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Summary of Review 

The research summary “Successful, Safe, and Healthy Students” presents the research 
background for the Obama administration’s proposals for comprehensive, community- wide 
services in high- poverty neighborhoods, extended learning time, family engagement and safe 
schools. While these policies have broad and common-sense appeal, the research supporting the 
particular policies proposed by the administration is weak and poorly presented in the research 
summary. As promising as community- wide services may be, a broad research base does not yet 
exist concerning how to make them successful. The research on extended learning time is also 
inconclusive. Family involvement is crucial to education, but the evidence for a causal link 
between student achievement and the type of parent involvement discussed is ambiguous and 
suspect. The proposals for safe schools boil down to increased local flexibility and increased 
gathering of survey data, neither of vdiich can be expected to improve outcomes. Together, the 
administration’s proposals would reguire an extensive financial commitment in order to be fully 
implemented, but the scope and source of these funds is not explained. Overall, the evidence 
provided is not sufficiently strong to justify the programs they champion. While the research 
summary adeguately documents problems, a wiser course for public policy would be a carefiilly 
structured set of pilot stuches to sharply and accurately identify solutions. 
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I. Introduction 

In March 2010, the Obama administration released a Blueprint outlining its proposals for 
reauthorizing the Elementaiy and Secondaiy Education Act (ESEA)dIn May 2010 theU.S. 
Department of Education (USDOE) followed with a set of six documents, offered as “research 
summaries” supporting the administration’s plans. ^ 

The fifth of these six research summaries, titled “Successful, Safe and Healthy Students” (SSHS), 
is the focus of this review. ^ The SSHS research summary dtes some of the large body of research 
pointing to the importance for educational outcomes of community poverty.^ Responding to 
related needs, the administration lays out a set of proposals divided into three sections, 
containing four categories of proposed policies: ( 1) Promise Neighborhoods (comprehensive 
services); (2) 21^* Century learning communities (extended learning time); and “Successful, safe 
and healthy students,” which includes both (3) family engagement and (4) increased flexibility 
and use of data to target health and safety needs. This review considers each of these four in 
turn, examining the strength of the research dted, how well that research represents the larger 
research base, and how well it supports the Blueprint’s policy proposals. 

Clear patterns emerge across the four categories of proposals: the research summary adeguately 
documents problems but generally stops far short of presenting meaningful research in support 
of the proposed policy; the evidence cited to support the administration’s proposals does not 
always actually support them; and unfavorable evidence tends to be omitted. These and other 
issues are described in greater detail in this review. 



II. Promise Neighborhoods 

Polides arB proposed to provide comprehensive services (including parenting education, 
preschool, health, and sodal and community development services) to children and 
families in very poor neighborhoods, with the goal of reducing achievement gaps between races, 
ethnic groups, and social classes. The most prominent proposal is the creation of Promise 
Neighborhoods nationwide based on the Harlem Children’s Zone (HCZ) model. Unfortunately, 
vdihe SSHS dtes some important research suggesting the benefits of HCZ, it fails to consider 
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adequately information on costs or the evidence that this approach may not be as effective as 
suggested by media reports. 

The only rigorous study of HCZ to date has failed to find any evidence linking improved 
achievement directly to its community and neighborhood components.^ This research is best 
understood as preliminary and ongoing, but it provides little support for a major scaling up of 
the HCZ approach. Even the HCZ’s early- parenting and preschool programs were found to be 
ineffective in isolation (apart from combined effects with other interventions), although 
abundant evidence exists elsewhere demonstrating that early- parenting and preschool programs 
have been found to raise achievement compared to the status quo.® 

The only positive academic effects found were associated with attendance at the HCZ “Promise 
Academy” charter schools— vdiose students also received the full panoply of comprehensive 
services. Therefore, the researchers could not attribute these results to the schools in isolation. 

Significantly, the cost of the HCZ (and other proposals) is not discussed in SSHS. Other 
research, however, has suggested that the costs of comprehensive services similar to those 

The only rigorous study of Harlem Children’s Zone to date has failed to 
find any evidenee linking improved aehievement direetly to its eommunity 
and neighborhood eomponents. 

proposed can add considerably to the cost of education even when they are not particularly 
effective.'^ The benefits of such programs might well outweigh the cost (relative to investing in 
other policy options), but there is no way of knowing without an accurate cost estimate. 

Notwithstanding these evidentiary limitations, the administration’s Promise Neighborhood 
proposal seems to be based entirely on the sparse HCZ evidence. In fact, a broader review of the 
research evidence raises important questions. While ample research points to the educational 
(and broader) importance of addressing out-of-school factors that affect children and families,® 
research on programs that move families from neighborhoods of concentrated poverty, for 
example, casts doubt on remedies focused on a direct causal link to concentrated poverty. ^ 
Similar doubts are raised by studies of preschool programs providing comprehensive services to 
young children.® This relative weakness in the research base does not necessarily argue against 
expanding and studying pilot “Promise Neighborhoods” efforts, but it is an argument for sound, 
complete examinations of research in advance of policy formation. 

A more balanced review than that presented in the SSHS research summary might also highlight 
the value of high-quality early childhood education programs. Such programs have been found 
to promote increased achievement and school success for disadvantaged children, yet they 
receive no mention in SSHS. 
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III. Expanded Learning Time 



The SSHS summaiy presents evidence in support of increasing the amount of time for 
instruction by adding hours to the school day or days to the school year. The stated approach 
also “[g]ives priority to programs that comprehensively redesign and expand the school day or 
year to increase time for academics or enrichment activities and provide time for teachers to 
collaborate and improve their practice.” In support of this recommendation, a variety of findings 
from empirical studies are briefly mentioned: 

1. U. S. students attend school fewer days per year than those in nations with which the U. S. 
competes economically and that outscore the U.S. on the Trends in International 
Mathematics and Science Study (TIMSS) test.^^ 

2. “Programs that lengthen the school day and year are a promising approach to improving 
student outcomes.”^ 

3. “Preliminary research indicates that expanded learning time can increase student 
achievement. ”14 

There is less here in empirical support for the policy than meets the eye. International 
comparisons are an unsound basis for making judgments about schooling in the U.S. Many 
influences other than the length of the school year differ among the countries. Further, surveys 
such as TIMSS have been justifiably criticized by researchers on the grounds of language 
differences (even in math and science guestions), inconsistencies in samphng students across 
nations, and differing ages of students tested among nations, These are just a few of the 
reasons vdiy experts believe that heavy reliance on TIMSS and other international comparisons 
in determining U.S. education policy is a seriously flawed strategy. 

Regarding the second point, the dted Traphagen and J ohnson- Staub reporti'^ offers only weak 
evidence for policy formulation. It is anecdotal, providing no data on the purported benefits of 
expanded learning time and student outcomes. 

On point three, the Farbman and Hoxby, Murarka, and Kang reports are the only meaningful 
sources dted for the daims about student achievement and expanded learning time. Farbman 
wrote, “ we determined a statistically significant (p< .0 1) moderate correlation for Grades 7 and 
10 in both math (r=.29 and r=.41, respectively) and ELA (r=.31 and r=.43, respectively). No 
significant differences were foimd for other grades.”!^ These tepid findings were all but retracted 
vdien the author subseguently noted that “Analyses do not suggest a causal relationship and do 
not control for any other school- or student- related factors (e.g., demographics)” (p. 25 ). 

Hoxby and her assodates performed a survey of charter schools in New York Qty in vdiich they 
related the lengths of the school day and the school year to student achievement. In their 
regression analysis, a longer school year was assodated with higher achievement. A longer 
school day may also be important, but since the schools in the database with longer days also 
tended to be those with longer year^, the longer day does not show up as a predidor vdien both 
variables are induded in the regression eguation (p. V-3). Interestingly, this day-versus-year 
distinction emerges in other research and thus is probably more than an artifact of the Hoxby 
data. Research on longer days has shown few if any benefits for student achievement. Research 
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on longer years has shown that v\iiile small increases in the number of days students attend 
school reveal no benefits for student learning, large increases in the length of the school year— 
for example, extensive summer school attendance— are more promising. 20 Moreover, Hoxby and 
her colleagues, like Farbman, expressly state that they make no claims about causes. 

The other source offered in support of expanded instructional time is a brief write-up of the 
Massachusetts Expanded Learning Time Initiative. This citation is curious. The research 
summary says that six of the seven middle schools in the study more closely approached the 

Research on longer days has shown few if any benefits for student 
achievement. 

state average in math (that is, they narrowed the existing gap), and five of the seven schools did 
so in language arts. Yet such promising statements of findings seem at odds with the conclusions 
of the evaluation of the program’s second year (2007-2008) : 

As is common in the early stages of new school-level initiatives, analyses revealed few effects 
of ELT [Ejqranded Learning Time] on student outcomes for the first cohort of ELT schools. 
For six of the eight Massachusetts Comprehensive Assessment System (MCAS) outcomes, 
there was no statistically significant effect of ELT on student MCAS performance in the first 
or second year of ELT implementation, controlling for student demographics, prior 
performance, and matched comparison schools’ performance. 21 

The weak support for the administration’s preferred policies given by the cited research is 
amplified by the omission in the SSHS research summary of key studies that should have been 
considered. For instance, the summary overlooked a meta- analysis by Levin and Glass on 
expanded instruction time that carried out a thorough cost-effectiveness analysis. Levin and 
Glass found that increasing the length of the school day for instruction was the second- least 
cost-effective of four interventions aimed at increasing student achievement (the others being 
cross-age tutoring, computer- assisted instruction, and reducing class size). 22 Increasing time for 
instruction was superior only to class-size reduction. Other evidence is mixed regarding the 
effects of a longer school day and after- school programs on academic achievement. 22 Moreover, 
a broader review of this evidence might have prompted consideration of a wider range of 
schooling outcomes— including habits, attitudes, social behavior, delinquency, and crime. 2^ 



IV. Family Involvement 

SSHS daims that “Many researchers agree that family engagement in their child’s education is 
positively assodated with increases in student academic achievement.”25 SSHS further daims 
that “school- sponsored family engagement programs may indirectly improve student 
achievement by increasing families’ expectations or improving parenting skills related to 
education.”26 
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Notwithstanding the careful wording of these claims, the evidentiary basis supporting the 
administration's policy proposals regarding family involvement is remarkably weak, and the 
research summary’s use of this evidence is partial and misleading. Of four recent major reviews 
of the literature on the effects of parent involvement on students’ academic progress (three 
meta- analyses and one narrative review), two are dted in the research summary^^ and two are 
not. 28 Following a narrative review of the literature, Mattin^y et al. (2002)— not dted by the 
USDOE— conduded that no influence of parent involvement on student achievement could be 

The non-experimental evidence for a causal link between parent 
involvement and student achievement is ambiguous and suspect, and the 
experimental evidence is essentially non-existent. 

found. Fan and Chen (2001), which is dted but poorly explained, found the literature on parent- 
involvement effects to have “considerable inconsistencies” (p. i) and concluded that “parental 
aspiration/ expectation for children’s education achievement has the strongest relationship [to 
measured achievement], whereas parental home supervision has the weakest relationship” (p. 

1) . This finding is echoed in the research discussed below and is particularly noteworthy since it 
provides a due to the potential invalidity of the research summary’s implidt daims about parent 
involvement produdng student achievement. 

The most extensive reviews of research on parent involvement are two meta-analyses: J eynes’s 
examinations of parent involvement at the elementary school level (2005) and the secondary 
school level (2007). The earlier, elementary study is dted; the later, secondary study is not. 
Although these meta- analyses led J eynes to condude that parent involvement increased student 
achievement, when viewed more critically they actually bring the validity of that conclusion into 
guestion. With one exception, all 90 studies integrated in Jeynes’s meta-analyses are non- 
experimental, one-shot ex post facto surveys in vdiich guestiormaire data from parents are 
correlated with student achievement measures. All that is actually revealed is a correlation of 
parent involvement (usually “aspirations” or “expectations for achievement”) with students’ 
measured achievement. Yet parents’ expectations may be influenced by their children’s 
achievement to at least as great an extent as their expectations cause increased achievement. 
This is suggested by stronger corrdations between aspirations/ expectations and achievement 
than between students’ achievement and actions taken by the parents in the home. 29 With just 
one guasi- experiment identified by J eynes in vdiich parent involvement is a manipulated 
variable, the evidence base seems insuffident to support a major policy initiative. 

The USDOE also dtes the study by D’Agostino and his colleagues^o in support of the benefidal 
impad of school- sponsored family- engagement programs. After controlling for “pre-reading 
achievement,” the largest portion of variance in reading achievement was accounted for by 
“parents’ aspirations for the child.” Once again, this finding raises the issue of ambiguous 
direction of causality. Do parental aspirations cause high achievement or does high achievement 
cause high parental expectations (or are they mutually reinfordng)? In addition, these 
researchers found no relationship between reading performance and any of the parent- 
involvement variables of “school- based collaboration,” “partidpation in school governance,” or 
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“volunteering at school.” This study is consistent with the larger body of research in this area: 
the non- experimental evidence for a causal link between parent involvement and student 
achievement is ambiguous and suspect, and the experimental evidence is essentially non- 
existent. 



V. Data to Improve Health and Safety 

The SSHS research summary also calls for “using data to improve students’ safety, health, and 
well-being, and increasing the capacity of states, districts, and schools to create safe, healthy, 
and drug- free environments.” The core idea— using data to inform policy— is undoubtedly 
sensible, but the particular proposals and the research support provided are more troubling. 

The USDOE argues in favor of three policies: ( 1) approaches that comprehensively improve 
school climate by improving safety and health; ( 2 ) the use of school- climate surveys to collect 
“needs assessment data” to guide the allocation of resources and the implementation of effective 
programs; and (3) the use of a grant application system allowing states and districts to apply for 
a single “Successful, Safe, and Healthy Students” grant and then target funds based on unigue 
local needs. The core reform recommendation is based on an expansion of the collection and use 
of survey data. The administration’s budget proposal seeks $410 million to implement “climate 
measurement systems” to be used to target resources to (a) improve school safety and 
strengthen family and student engagement, and (b) improve students’ physical and mental 
health.31 

The research summary implicitly limits the focus of “Successful, Safe, and Healthy Students” to 
only three key issues: school violence, the use of drugs (including alcohol), and obesity. ^2 The 
summary dtes some rosearch providing statistics to demonstrate that these problems exist, and 
the brief prosentation is sufficient to justify these issues as deserving of a policy rosponse. We 
note, however, two omissions. First, nothing is provided in the rosearch summary concerning 
school violence trends. And in fact, rates of most categories of violence in school are generally 
continuing a long decline. wg also note that lesbian, gay, bisexual, and transgendered (LGBT) 
students are particularly at risk for school violence, but the research summary never mentions 
this, nor does the administration’s Blueprint policy focus on the problem. 

Further, as noted in the research summary itself, use of marijuana and tobacco are both 
trending downward, and there are successful interventions currently being used. So vdiile the 
research summary makes the case that violence and the use of drugs and alcohol are 
widespread, it falls short of making an evidence- based case documenting the key policy 
guestions: which programs are currently in use, how effective are they, and is policy in need of 
change. The change argument relies on a 2007 USDOE report concluding that the current grant 
stmcture does not adeguately target the neediest schools. That may indeed be the case, but 
given the positive trends, more justification for change would be helpful— particularly for the 
specific changes called for in the Blueprint. 

In contrast, the trends as well as the rates of obesity are clearly troubling. What is missing from 
the research support provided by the USDOE is evidence supporting the Blueprint’s focus on a 
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school- specific response, rather than a broader program to address obesity at a societal level. No 
research citations are provided concerning school- level programs. Undoubtedly, schools can 
play a positive role through education, the provision of healthy breakfasts and lunches, the 
elimination of junk food sales and advertising on campus, and the provision of good 
opportunities for exercise during the school day.^^ However, it is not at all dear from the 
Blueprint that grant money would be spent in ways supported by research. Moreover, the 
obesity problem and solutions to it go far beyond the role of schools. Again, none of this 
evidence is presented in the research summary, and none of these issues is addressed. 

The central idea behind school- climate surveys seems to be that these state- level and district- 
level surveys can determine (a) the nature of spedfic needs, and (b) where to allocate resources. 
The survey data could also be used longitudinally to determine the effectiveness of resource 
allocation and of particular programs. The use of school- dimate- survey data for these purposes 
has not been validated in any of the sources dted, however. While the primary survey 
instrument contemplated for this use has apparently been found to yield useful information 

Unfortunately, the researeh presented by the administration in support of 
its proposals generally stops far short of offering meaningful findings or 
illustrating useful poliey direetions. 

about key school- climate constructs, the question here is one of “use validity”: on vdiat spedfic 
bases vdll policy be grounded, and will the survey results provide the information needed to 
make sound policy dedsions? Neither the Blueprint nor the research summaiy indicate that the 
surveys have been, or will be, piloted and studied prior to being scaled up. Instead, the approach 
appears to be widespread adoption prior to study. This places a great deal of faith in a survey 
instrument that has apparently not been used for high- stakes purposes or been validated for 
such uses. It also raises the concern, generally expressed as Campbell’s Law, that the high stakes 
themselves will corrupt the survey. The cjuestion arises as to why multiple tools and 
measures— beyond the survey— are apparently not being considered.^s 

Overall, vdiile safety and health issues are important and while better policy approaches should 
be sought, the case is never made that increased local flexibility informed by survey data will 
result in more successful policies and programs. 



VI. Conclusion and Recommendations for Policy 

The research summary proposes laudable ideals by viewing schools as integral parts of the 
community and by advocating increasing family involvement in schools, extending leanring 
time, and promoting health and safety. Few would disagree with these ideals, and the summary 
does an adecjuate job of raising concerns in these aroas. 

Unfortunately, the research presented by the administration in support of its proposals 
generally stops far short of offering meaningful findings or illustrating useful policy directions. 
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The evidence cited to support the administration’s proposals does not in fact always support 
them, and imfavorable evidence tends to be omitted. 

Extended learning time, for example, is a notion with far greater nuance than is presented. 
Greater school safety is a laudable goal, but the research summary never makes the case for how 
added local flexibility or the collection of additional data will improve the situation. Family 
involvement is certainly vital to social well-being, but little definition is provided of what that 
involvement entails. Nor is an explanation provided of how the changes would be accomplished, 
and there is no explanation of the true nature and limitations of the research evidence. Early 
education is well-established in the literature, and research also points to enormous out-of- 
school needs in low- income communities; however, specific wrap-around support mechanisms 
such as HCZ are not as well studied and lack the same sort of research support. Furthermore, 
the cost of many of these proposals is significant, and there is no indication of how funding 
would be provided. 

The mismatch between the administration’s avowed— and sometimes demonstrated— 
commitment to evidence- based policymaking and the Blueprint policymaking process raises the 
guestion of vdiether a broader and more balanced review of the SSHS evidence might lead the 
administration to a different set of policy proposals. Such a balanced review should, we think, 
drive a carefully evaluated test of multiple promising approaches to addressing the identified 
needs. 
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